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The problem addressed in Professor Edwards' letter to 
the Director and the Director's response regarding the 
treatment of subjective probabilities in estimates reflects 
a chronic problem in the Intelligence Community. Approximately 
10 years ago the President's Foreign Intelligence Advisory 
Board noted the inadequacy of the Intelligence Community's 
use of existing quantitative techniques in the development 
of intelligence estimates. Subsequently, the Information 
Science Center was established to provide training in 
applicable methodologies. The utilization of these method- 
ologies has been relatively slow, although the attendance in 
Information Science courses has risen from approximately 50 
students per year in 1970 to over 600 per year at the present 
time . 

The use of quantitative values to clarify verbal statements 
of probability has been most advanced in the Defense Intelligence 
Agency. In general, although this has posed some difficulty 
for the analysts, the reception of Defense Intelligence 
Agency products with numerical probabilities has been very 
favorable, particularly by major commanders and the JCS. It 
is the opinion of Vice Admiral Robertson, USN, the Vice 
Director for Production, DIA, and Brigadier General Williams, 

USA, the Deputy Director for Estimates, DIA, that the analysts' 
difficulties have been ameliorated by various training 
programs in the use of probabilities and statistics. It has 
also been the experience of these senior officers that use 
of numerical probabilities, even in a simplistic form, 
encourages the analyst to think in terms of alternatives and 
their relative likelihoods. 

Relevant Training 

Subjective Probability Assessment Training (SPATr) 

Numbers can serve as an objective standard to supplement 
the words in intelligence estimates. The problem is not in 
developing a technique to improve communication; we have 
one. The problem is to implement it. An exercise developed 
from a 1968 Department of Defense research program has been 
used by the Information Science Center in the CIA Office of 
Training since the early 1970s. The Subjective Probability 
Assessment Test (SPATe) offers students an opportunity to 
use numbers as a communication device and to receive feedback 
on how the numbers they used compare with their actual 
performance. Approximately 2,000 Intelligence Community 
analysts and managers have taken the SPATe since 1976. 

There is clearly room for improvement in the way analysts 
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use numbers to express their uncertainty. We believe it is 
possible to train individuals to express their probability 
estimates more accurately. A program for Subjective Probability 
Assessment Training (SPATr) on the CIA's VM system has been 
available for seven months. 

Statistical Concepts for Analysts and Managers (SCAM) 

This course was established in April 1977 at the request 
of DIA. It is designed to acquaint analysts with the basic 
concepts and application of probability and statistics. 

Earlier, DIA had required the use of probabilities in Defense 
Intelligence Estimates and in current reporting. This 
course was considered necessary to improve the analyst's 
ability to fulfill that requirement. A total of 210 students, 
134 from DIA, 44 from CIA, and 32 from other agencies of the 
Intelligence Community, have completed the course. In 
addition to basic concepts of probability, the course includes 
such topics as Bayesian Analysis, confidence intervals, and 
subjective probability. Students also learn how to use 
computer programs to assist in problem-solving. 

We have found that some people- -because of mathematical 
background, dislike for numbers, or whatever the reason- -do 
not do well in this course. A limited amount of pre-testing 
suggests that not all analysts will accept or be able to use 
numerical probabilities well. 

The SCAM course is available on videotape. 

Decision Analysis Course 

The process of decision analysis involves the decomposition 
of a decision problem into clearly defined components so 
that all options, outcomes, values, and probabilities are 
depicted. The Information Science Center has presented 
short overviews of the techniques of decision analysis in 
several courses, and the first course devoted exclusively to 
this technique was presented 4-8 December 1978 at the ISC by 
Decisions and Designs, Inc.' This course is scheduled three 
times yearly. 

Over the past few years there have been a number of 
applications of decision analysis in intelligence. In the 
DoD, for example, it has been used to study evacuation 
situations, to structure Program Objectives Memoranda (POMS), 
and to analyze indications and warning situations. Recently, 
a hierarchical inference model was installed on the CIA's VM 
system for use by CIA analysts. Because the techniques of 
decision analysis provide a simple but useful framework 
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within which components of a decision problem can be analyzed, 
they can be used to both understand and communicate to 
others the results of analysis. 

Simulation Techniques 

The preceding techniques are also important because of 
their potential contribution in the development of simulation 
models. Simulation models are uniquely valuable because 
they can be used to quickly assess the impact of various 
policy alternatives. This addresses directly the nature of 
the problem with which Professor Edwards and Admiral Turner 
are dealing. Training in simulation model building, the 
System Dynamics course, has been presented by the ISC for 
five years . 
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